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DETAILED ACTION 

1. This communication is in response to Amendment filed 01/06/03, claims 1-2, 4-8, 18-26, 28-29 
and 3 1-42 remain pending in this application. 

2. On the remarks of the above-mentioned amendment, Applicant indicated that a Declaration under 
37 CFR §1.131 to swear behind the Bar et. al. reference will be filed as soon as available. This noted. 
Applicant was extended a courtesy telephone call (Kelton, M. Reg. No. 44,182 on 02/06/03) to determine 
if the said declaration had been filed, in order for it to be fully considered prior to issuing hereby office 
action. Applicant indicated that due to the numerous inventor's of instant application to this date such 
declaration has not been completed and therefore has not been filed. 

3. Quotation of 35 U.S.C § 103(a) which forms the basis for all obviousness rejections set forth in 
this Office action may be found in previous action; 

4. Claims 1-2, 4-8, 18-26, 28-29 are 31-42 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Smith et. al. (Smith) U.S. Patent No. 6,128,649 in view of Clapp et. al. (Clapp) U.S. 
Patent No. 5,802,281 in further view of Bar et. al (Bar) US. Patent No. 6,122,665. 

Regarding claims 1 and 21, specifically in regards claim 1, Smith teach features of the invention 
substantially as claimed, Smith teach a system/method in a network conferencing environment (see 
abstract, col 1 /lines 64-67) for delivering a plurality of video or audio media data type signals (see Figs. 
1 -5a, audio and/or video media streams, col 3/lines 20-35) the system comprising; 

transmitting a set of media data streams on to the network, set of media data streams generated 
from the plurality of video or audio type signals (abstract, video streams, distributing audio/video streams 
across the network, col 1/lines 53-67); 

transmitting means include means for removing silences from said data streams of the audio 
signals transmitted by the transmitter (identifying silence stream, col 9/lines 5-9, removing said identified 
streams from data audio transmission stream by closing audio channel from originator); 

a receiver for receiving the set of data stream from the network (col 20/lines 1 1 -col 21/line 25, 
transmission/reception processing modules), the receiver including a selectively routing, filtering or 
separating media streams (i.e. demultiplexing) means (multiplexing (1) means, col 1/lines 53-62, Fig. 21) 
for dynamically selecting a subset of the set of data streams (dynamic selection (13), means, col 6/lines 
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49-col 7/line 26, dynamic selection of multiple media streams, see abstract, and multiplexing means col 
27/lines 31-55); 

wherein received media data stream type is from respective first/second respective conference 
participant (Smith: participants receiving mixed or selected (audio/video type) subset media data streams 
associated with sent audio/video type media data stream over (2) network col 1 /lines 53-col 2/line 6, Figs. 
2-3, receiving audio/video means (26, 32) of Fig. 5b and rendering, i.e. displaying or outputting said 
selected subset) and recovering means for recovering the data streams into audio or video signal at one 
receiver (end-system) receiving the set of data streams (recovery means, col 23/lines 50-61); and 

two or more receiver media data stream (payload) handler modules (col 20/line 1 1-col 21/line 25, 
transmission/reception processing modules, reception audio/video process modules, col 7/lines 35-48); 
specifically in regards to claim 21; 

wherein said means for selectively routing, filtering or separating media streams (i.e. 
demultiplexing) is configured or conformed to handle, process or transport media streams to/from the a 
network media using real-time transport protocol (RTP) (col 21/lines 26-53, receiving/transmitting RTP 
media streams); 

said selectively routing, filtering or separating media streams i.e. demultiplexing, including 
routing one or more RTP data streams of the portion based on data type (selecting a particular type of 
media streams from the plurality of media streams, col 1 /lines 53-col 2/line 6); 

two receiving (media-in portion 20, col 7/lines 35-39) including receivers coupled to said 
demultiplexing means for handling routed data streams (first reception means col 7/lines 58-67, having 
decoding (28) means, and second reception means col 8/lines 12-22); 

two decoder modules coupled to the demultiplexing means for decoding routed data streams (col 
20/lines 1 1-30); and 

a rendering means coupled to the decoder for playing back one RTP data stream (col 20/lines 3 ! - 



transmitting (44, 43) means (col 8/lines 23-36, Fig. 5a (21), Fig. 5b (31, 36, 43)) for transmitting 
a set of data streams comprising audio (43) signals onto a network (col 8/line 56-61 means for 
transmitting a subset of audio streams) 

wherein transmitting means include means for removing silences from said data streams (means 
for identifying silence stream, col 9/lines 5-9, removing streams from data audio transmission stream by 
closing audio channel from originator), 

however Smith does not explicitly teach wherein said two decoder modules for decoding two 
particular types of the data streams, 



48); 
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Clapp teaches a computer system comprising two or more receiver payload handler modules and 
two or more corresponding decoder modules for handling and decoding two or more types of data (Fig. 5, 
(elements 150, 170, 102, 104, 70)), col 5/lines 1-5, 20-22), disclosing a system/method related to network 
conferencing peripherals adapted for stand alone use/operation with a host computer system (col 1 /lines 
7-13, col 6/lines 8-20). 

Although the above-mentioned prior art teach dynamically selection a subset of the subset of data 
stream, prior art does not explicitly teach wherein the selection is based on a source identifier and a 
payload type; 

Bar teaches Bar teaches means for demultiplexing means operatively coupled to one decoders for 
routing data to one of the decoders based on said data type and source identifier (col 6/lines 5-40, col 
8/lines 64-col 9/line 7, elements 28, 32 of Fig. 2, data type determination Fig. 3 A, step 3 A, col 9/lines 33- 
48, synchronizing according to type, col 13/lines 3-31, filtering based on the type and selecting based on 
the source identifier, col 3/lines 50-col 4/line 29); 

Lt would have been obvious to one ordinary skilled in the art at the time the invention was made 
to modify existing system with means for configuring two receiver payload handler modules and two 
corresponding decoder modules for handling and decoding two or more types of data, as taught by Clapp, 
motivation would be implement an audio/video communication peripheral of substantially reduced cost, 
compact and portable for effectuating video/audio conferencing which includes a plurality of decoders for 
standard types of data streams. Further, multiplexing based on the payload type and source identifier, as 
taught by Bar, motivation would be configure filtering means adaptable to process different types of 
audio/video media streams wherein based on the filtering capabilities a suitable type associated to the 
different types of media enable the selection of the decoders and rending modules to be used on the 
conferencing system routing the different data streams accordingly in real-time. 

Regarding claims 2, 4 and 8, combined teachings as discussed above, however do not explicitly teach 
wherein one of the payload handler modules handles audio G.71 1 data and another handles G723.1 data 
and one decoders for decoding audio G.711 data and another decoder for decoding audio G.723,1, 
comprising means for demultiplexing coupled to the two receiver payload handler modules for routing 
data to one of the receiver payload handlers based on data type, further comprising demultiplexing means 
coupled to the one decoders for routing data to one of the decoders based on data type; 

Bar teaches means for demultiplexing means operatively coupled to one decoders for routing data 
to one of the decoders based on said data type (col 6/lines 5-40, col 8/lines 64-col 9/line 7, Fig. 2 (28, 32), 
col 9/lines 33-48); disclosing a computer system (Figs 1-7) comprising two or more payload modules for 
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receiving routed data stream, handlers coupled to the device (28) configured a process applied to 
combined multiplex signal for recovering signal combined within it and for restoring individual channels 
of the data type signals, demux to separate two or more combined signal, i.e. demultiplexing a RTP data 
type signal stream; wherein one of the handler modules handles audio data type G.71 1 data and another 
handles audio data type G.723. 1 data and one decoder module, (Figs. 1-5) decodes audio data type G.71 1 
data and another decodes audio data type G.723. 1 data; separating stream means (38) coupled to the two 
receiver payload handler (28) modules for routing selected data to one of the corresponding relevant 
receiver payload handlers based on data type (col 9/lines 33-48, col 10/lines 21-47, selecting based on 
data type (60) for routing via corresponding respective channels to relevant received data handles (type 
base data selection means, col 2/lines 32-38, receiving selected data type means, col 3/lines 24-29, 50-61, 
routing by data type means, col 6/lines 5-40, routing data to decoders based on data type means, Bar: Fig. 
3a, receiving/separating means (24) data type streams and routing to relevant component associated with 
data type stream, Fig. 2 (28), col 10/lines 21-47, col 8/lines 5-27, 50-56, col 8/line 66-col 9/line 7 routing 
data protocol type channel to decoders (104, 102) including RTP data type stream); Bar teaches Bar 
teaches means for demultiplexing means operatively coupled to one decoders for routing data to one of 
the decoders based on said data type and source identifier (col 6/lines 5-40, col 8/lines 64-col 9/line 7, 
elements 28, 32 of Fig. 2, data type determination Fig. 3A, step 3A, col 9/lines 33-48, synchronizing 
according to type, col 13/lines 3-31, filtering based on the type and selecting based on the source 
identifier, col 3/lines 50-col 4/line 29, based on data type and source identifier: col 6/lines 5-40, col 
8/lines 64-col 9/line 7, elements 28, 32 of Fig. 2, data type determination Fig. 3 A, step 3 A, col 9/lines 33- 
48, synchronizing according to type, col 13/lines 3-31, filtering based on the type and selecting based on 
the source identifier, col 3/lines 50-col 4/line 29). 

Regarding claim 5, the combined teachings as discussed above however do not explicitly teach for mixing 
an audio stream operatively coupled to the two or more corresponding decoders. 

Official Notice {see MPEP § 2144.03 Reliance on "Well Known" Prior Art) is taken that a mixer 
was old and well known in the Data Processing art. It would have been obvious to one of ordinary skill in 
the art at the time of applicant's invention to include a mixer for mixing audio stream, motivation would 
be to render a composite audio signal to the user. 

Regarding claim 6, the combined teachings as discussed above, further including a means for rending data 
stream obtained from said decoders; media rendering module operatively coupled to the one or more 
decoders (Clapp: Fig. 5, (122/140, 220, 74)), a high-speed output interface provides connectivity with the 
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separate host computer system for coordinating, in cooperation with video conferencing application 
software operating thereon, the presentation of local and remote video, abstract, col 4/lines 38-45, col 
6/lines 21-43). 

Regarding claim 7, the combined teachings as discussed above, wherein one or more of the payload 
handlers includes: means for reassembling or combining two or more data packets, means for reordering 
data packets (Clapp: col 21/lines 20-26). 

Regarding claims 18 and 22, the combined teachings as discussed above, further teach a method of 
conducting a network conference with two or more computer systems (Clapp: col 4/lines 37-34, Fig. 1), 
comprising: 

receiving audio or video data from first and second computer systems (Clapp: Fig. 5, col 80, 78 

and 82); 

determining the type of the audio or video data from the first computer system (Clapp: Fig. 5, 
(200) routing/separating means, col 8/lines 59-67 and 22/lines 57-59, 9/lines 8-18 and 21/lines 20-26), 

routing the audio or video data from the first computer system to a first decoder based on the 
determination of the type of audio or video data (Bar: col 8/lines 25-col 9/line 7), 

determining the type of the audio or video data from the second computer system (Bar: col 9/lines 
33-48, col 10/lines 21-65); and 

routing the audio or video data from the second computer system to a second decoder based on 
the determination of the type of audio or video data: (Bar: col 8/lines 25-38, col 9/line 7); 

further monitoring incoming audio or video data for each of a plurality of conference parties for 
active or inactive status; (Bar: col 1 1 /lines 10-50, col 7/lines 21-25); 

means for examining incoming (i.e. monitoring) audio or video data for each of a plurality of 
conference parties for active or inactive status (i.e. initiated session (Bar: col 1 1 /lines 10-34, determining 
packet data status, col 3/lines 50-61 for an active status); 

monitoring means further comprising monitoring incoming audio or video data for a beginning, 
starting (i.e. new) participant, col 11 /lines 10-50, wherein communication session include speaking 
participants, col 9/lines 1 1-20); 

wherein additionally Smith teach means for monitoring Fig. 5B, (33-35) incoming audio data for 
each of a plurality of conference parties for active or inactive status (Smith: means for determining 
whether one media streams is active, col 3/lines 61 -col 4/line 11, stream activity monitoring/detection 
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means (33), determined stream activity state changes of a stream on the corresponding audio stream, col 
9/lines 10-57, determining which said streams are silent or less active, Figs. 7-8); 

monitoring means further comprise monitoring audio data for a new speaker: (new stream activity 
detection means, col 10/lines 44-col 1/line 19, stream activity associated with conference participant's 
GUI event (60, 70), col 9/lines 10-57); 

replacing audio data having the inactive status with data of the new speaker; (Smith: means for 
substituting a set of data from another (third) conference participant with data set from a respective 
determined inactive participant, comprising means for determining (150) the most silent stream to be 
replace, wherein in response to a positive determination replacing (dropped) said most silent stream with 
said (third) stream, col 10/line 43-col 1 1/line 19, replacing a silent stream with a another (third) data set 
associated with another participant, mean for detecting most recent speaker and performing substitution 
steps, col 1 9/lines 3-45, Bar; means for monitoring means further comprising monitoring incoming audio 
or video data for a beginning, starting (i.e. new) participant, col 1 1/lines 10-50, wherein communication 
session include speaking participants, col 9/lines 1 1-20). 

Regarding claim 19, the combined teachings as discussed above, the method of further comprising: 
decoding the audio or video data from the first and second computer systems (Clapp: col 9/lines 47-60, 
audio: col 22/lines 59-62) computer-readable medium comprising a first set of computer-executable 
instructions by which said decoders operate (Clapp: col 9/lines 47-60 (integrated circuit)), and rendering 
the audio or video data from the first and second computer systems (Clap: abstract, col 4/lines 38-45, col 
6/lines 21-43, Fig. 5). 

Regarding claim 20, the claim is substantially the same as claim 2, same rationale is applicable. 

Regarding claim 23, the combined teachings as discussed above, teaches a network conferencing system 
comprising: 

an separating/filtering means, i.e. demultiplexing means adapted to handle RTP data stream (RTP 
demultiplexer) for receiving and routing one or more RTP data streams based on data type; (Bar: 
separating stream means (38) coupled to the two receiver payload handler (28) modules for routing 
selected data to one of the corresponding relevant receiver payload handlers based on data type (col 
9/lines 33-48, col 10/lines 21-47), 

selecting based on data type (60) for routing via corresponding respective channels to relevant 
received data handles (type base data selection means, col 2/lines 32-38, receiving selected data type 
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means, col 3/lines 24-29, 50-61, routing by data type means, col 6/lines 5-40, routing data to decoders 
based on data type means, Bar: Fig. 3a, receiving/separating means (24) data type streams and routing to 
relevant component associated with data type stream, Fig. 2 (28), col 10/lines 21-47, col 8/lines 5-27, 50- 
56, col 8/line 66-col 9/line 7 routing data protocol type channel to decoders (104, 102) including RTP 
data type stream); and a rendering module (Clapp: col 6/lines 60-64) coupled to the decoder for playing 
back of said one RTP data streams). 

Regarding claims 24-26 and 32, the combined teachings as discussed above further teach means for 

receiving via a communication network a first and second set of data type from a first and second 
conference participants; (Smith: each participant (3) of a conference system sending audio/video type 
media streams over a communication (2) network, 

each participants receiving mixed or selected subset of (audio/video type) media streams (Smith: 
col 1/lines 53-col 2/line 6, Figs. 2-3, receiving (reception) audio/video means (26, 32) of Fig. 5b); 

selecting (Bar: filtering components (38, 40) determine media type col 9/lines 33-40) a subset of 
the plurality of audio media data streams (Bar: media data stream selected from a group of audio data, col 
2/lines 32-37, selecting by type col 3/lines 25-34) including media data streams of different media data 
types (Bar: col 6/lines 5-20, selected subset in accordance with different media data stream voice types, 
channeled per media data stream type, col 71ines 56-col 8/line 1 18-24); 

routing (Bar: filtering (24) passes selected, filtering by type and passing said selected type col 
7/lines 56-col 8/line 1 to corresponding relevant module (28) type, of Fig. 2), the selected subset of the 
plurality of audio media data stream to corresponding decoder modules (G.711, G.722, etc.,) in audio 
codec component (Bar: 102 of Fig. 2, col 1 3/lines 3 1-48) based on their media data stream type (Bar: Fig. 
5, steps 3-5, determined type, col 10/lines 21-47, media type including standard categories, including 
audio standards, col 6/lines 5-20); 

rendering, i.e. displaying/outputting said selected subset (Bar: Fig. 1, illustrating receiving means 
(16), selecting means (24), rending means (36, 34) media data streams, displaying audio and/or video, col 
5/lines 13-15)); 

decoding received-routed first/second type audio data stream in corresponding first/second 
decoder type (Bar: col 9/lines 33-48, col 10/lines 21-47, col 2/lines 32-38, col 6/lines 5-40, Fig. 3a, Fig. 2 
(28), decoders (104, 102)); 

determining whether one of the first and second set of data type media stream from said first and 
second conference participant is associated with an inactive conference participant (Smith: means for 
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determining whether one media streams is active, col 3/lines 61 -col 4/line 11, stream activity 
monitoring/detection means (33), 

selection means based on at least two media streams in response to determination means, said 
determination means are based on the detection of a conference participant's GUI event (Smith: 60 5 70 of 
Figs. 7-8), event is determined to relate to a state changes of a stream corresponding to a detection of 
activity on the corresponding audio stream (Smith: col 9/lines 10-57, disclosing means for determining 
which said streams are silent or less active, Figs. 7-8); 

responsive to said determination substituting a third set of data from a third conference participant 
with data set from respective inactive participant; (Smith: means for substituting a set of data from 
another (third) conference participant with data set from a respective determined inactive participant, 
comprising means for determining (150) the most silent stream to be replace, wherein in response to a 
positive determination replacing (dropped) said most silent stream with said (third) stream, col 10/line 43- 
col 1 1/line 19, replacing a silent stream with a another (third) data set associated with another 
participant); 

demultiplexing means operatively coupled to one decoders for routing data to one of the decoders 
based on said data type and source identifier (Bar: col 6/lines 5-40, col 8/lines 64-col 9/line 7, elements 
28, 32 of Fig. 2, data type determination Fig. 3 A, step 3A, col 9/lines 33-48, synchronizing according to 
type, col 1 3/lines 3-31, filtering based on the type and selecting based on the source identifier, col 3/lines 
50-col 4/line 29); 

Regarding claims 28, this claim comprises limitations that are substantially the same as combined 
limitation of claims 25 and 27, same rationale is applicable. 

Regarding claim 29, 31, this claim comprises limitations that are substantially the same as claim 26 and 
28, respectively, same rationale is applicable. 

Regarding claim 33, this claim comprises limitations that are substantially the same as combined claim 
27, same rationale is applicable. 

Regarding claims 34-36, the combined teachings as discussed above, further teach 

wherein the selected subset includes a first video data stream formatted according to a first 
protocol an a second video data stream formatted according to a second protocol, wherein the data 
streams in the selected subset are most recently activate data streams (Smith: col 1 /lines 63-col 2/line 2), 
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selection based on monitored activity, event detection data stream activity (Smith: col 19/lines 3- 
21, most recent audio data stream activity associated with a participant, col 19/lines 22-45, wherein the 
first and second sets of data streams are audio signal data of a multicast group of (e.g. dialogue) between 
two or more participants). 

Regarding claims 37-42, synchronization source identifier (Bar: col 13/lines 3-24). 

4. Claims 1, and 21 may additionally be rejected under 35 U.S.C. 103(a) as being unpatentable over 
Smith et. al. (Smith) U.S. Patent No. 6,128,649 in view of Clapp et. al. (Clapp) U.S. Patent No. 5,802,281 
in further view of Northlich, B. H.3.2 Centralized multipoint configuration, Feb. 1997. 

Regarding claims 1 and 21, substantially the same as discussed above (e.g. claims 1 and 21), herein 
incorporated by reference, same rationale is applicable, however although the above mentioned prior art 
teach dynamically selection a subset of the subset of data stream, prior art does not explicitly teach 
wherein the selection is based on a source identifier and a payload type; 

Northlich teaches means for demultiplexing streams (RTP) based on SSRC and payload type, as a 
clarification to the ITU Telecommunication Standard (H.323), to include streams of video and/or audio 
channels in a conference environment (see page 2). 

It would have been obvious to one ordinary skilled in the relevant art at the time the invention 
was made to include means for selecting a subset of the set of data stream based on a source identifier and 
a payload, as taught by Northlich, motivation would be enable a terminal to support simultaneous session 
in multiple data stream types to mix both audio and/or video in a conferencing system. 
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